Part of the problem is that people are set up to fail. I have a friend that is a professor at a college near where I live. He tells of students that are not prepared for college that are admitted on the basis of being some under represented minority. The majority go home before the first semester is done. They then tell all their friends and neighbors how hard it was and don't bother.
There is a fair share that make it. Usually with huge amounts of help and tutoring, but they wouldn't have made it without the hand-holding. The problem is that if they get a job in the field, they won't be able to perform to the standard that would be expected. The results being what has been mentioned in previous posts.
The key would be to enroll them in trade schools or college programs that they can be sure of completing. This would boost the student's self esteem and prepare him for a productive future. As people are able to escape poor neighborhoods - or improve the ones they are in, others will be inspired to do the same.